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ABSTRACT §; v 

A study was undertaken to provide a s third analysis of 
implementation of evaluation requirements in vocational education and 
of their perceived effectiveness and utility. The sample included the 
total population of Directors of Vocational Education for c the United 
States and its territories; 50 directors completed the questionnaire. 
All respondents indicated they had. either fully implemented or were 
in the "process of implementing the mandated evaluation activities. 
Program improvement was the primary reason for adopting current 
evaluation practices. ''Employer feedback and placement of former 
students were identified as the most often identified indicators of 
program quality. Student f ollowup and employer assessment of former 
students were the most fully implemented evaluation activities. 
Assessment of student performance and of special needs services 
remained the least implemented activities. Respondents were generally 
satisfied with their staters system for evaluating vocational 
V programs. Th«?y also indicated a high level of suae fulness of 
\evaluation activities. Specific purposes for which they were used 
\rere making improvements, preparing accountability reports, and 
adding decision making. Recommendations for further development of 
federal policy were emphasizing evaluation's improvement function and 
keeping requirements for evaluation procedures consistent. (Study 
questions are provided with conclusions and evidence/responses. An 
approach for evaluation for and of improvement is suggested.) 
(TLB) » 
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introduction 



' Since the early 1960's, federal legislation for' education has made 
^reference to evaluation. The 1963 Vocational Education Act provided for 
using federal funds for state and local evaluations of vocational educa- 

> » ^ 

tion and all its subsequent amendments have increased the attention 
given to evaluation. The most recent, the Education Amendments of 1976 
(PL 94-482), introduced new requirements that have exceeded all those 
previous in both number and complexity. 

The National Institute of Education was charged by the 1976 amend- 
ments to conduct a comprehensive study of vocational education and was also 

* * » * 

directed to analyze "the means of assessing, program quality and effectives 
ness": Part of the NIE thrust focused on implementation of the act's. 

i * d 

evaluative requirements. This was. accomplished at two points prior to this 
study - in the spring of 1978 (CRC, 1978) and during the 1979-81 school 
year (Abt, 1980). The study reported herein^provides a third metering of 

evaluation. practice as of the spiring of 1981. This study involved a survey 

I 

of state directors of vocational education and was administered by the National 

/ 

Association of State pi rectors /of Vocational Education (NASDVE) with support 

from the Vocational Education /study of the National Institute of Education. 

: This study sought, to gajn.a third perspective to the implementation 

of -evaluation requirements and their perceived effectiveness and utility... 

Wore specifically, 'the study sought to answer the following questions: 

1. What do state directors of vocational education perceive 
to be the key indicators of program quality? 



/ 



2. How do SEA's /interact*^ th LEA's in the collection and use 

of evaluative thformati on? / 

/••*'. * -' 
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3. What is the perceived utility, of current evaluation 
practices in SEA's? ' ^ 

4. What factors facilitate and deter the initiation and conduct, 
of state evaluation practices? v 

5* Should the existing criteria for evaluation be. retained? 

• 6. What is the extent of implementation of current evaluation 
practices? 

7., What is the desired unit of focus for evaluation and 

program improvement? , ♦ 4 

* * * 

,8* . What suggestions do state directors have for writing new 
legislation? 

These questions focused the collection and analysis of information. 

The "General. Recommendation" and "Suggestion" at the close of this 
report are made by the authcfr, and do. not necessarily reflect the views of 
the NASDVE or the NIE, which commissioned this report; or its Vocational 
Education -Study. ^ 

method 



Sampl e 1 • * V 

The sample for this study included the total population of^Directors 
of Vocational Education for the United States and its territories. A 
total of 57 questionnaires were mailed and responses were received from 50. 
This calculates to an 88% return rate. Of the 50 respondents, 32 were state 
directors and 18-were SE*A staff members designated by the state directors. 
Instrument * 

\ K • ' • . % 

A' four-page questionnaire was used in the survey. This questionnaire. ) 

• \ ' ** ' ") 

incorporated two rating type items, eleven checklist items, one combination 



Procedure , , ^ 

The^questionnaires were mailed with a cover letter signed by the 
Executive Director of National Association of State Directors of 

Vocational Education. Also,. NASDVE .was responsible for the follow-up 

. - 

of non-respondents. All questionnaires were mailed on March 23, 1981, 

» • i 

and the last response was received on May 14, 1981. 
Analysis . '. 

The questionnaires were summarized using frequency counts, measures 
of central tendency, and a. measure , of dispersion. Questionnaire item^ 
responses were categorized and reported to parallel the study questions 
presented in the' introduction to this report. 

• • • / ' 

results 

The following pages presentrthe study questions, the questionnaire 

evidence that corresponds to each, and a series of judgments presented in 

" / ' 
the forrt of conclusions". ■ 



.Key Question 1 . 



What do State Directors <ff Voc-Ed 
perceive to be the key indicators of 
program quality? 




conclusions 



evidence (responses to survey question) 



Employer feedback and placement are identi- 
fied as the primary indicators of program, 
quality. These a re foil owed by instructional 
materials, job satisfaction, instructor 
performance, and student test performance 
(these- four were rated almost equally). 



14. Of the following criteria, which would you rely on 
as indicators of program quality? 

44 - Placement Level * 

25 - Condition of Equipment ' r 

36.- Instructor Performance 

37 - Quality Curriculum Materials 

47 - Employer Feedback 

37 Job Satisfaction i 

17 - School .Staff Morale ^ 

31 - Student Test Performance 

1 - Participation : 

1 - Serving Special Needs 

1 - Instructional Preparation 

1 - Facilities' 

X - Instructional Materials 

1 - Retention of Students 

. 1 - Advisory Committee Use ' 

1 - Guidance, Counseling and Placement 



. Key Question 2. 



How do SEA's interact with 
LEA's in the collection and use 
of evaluative information? 




conclusions 



evidence 



1. LEA's appear to have a high level of involve- 

* ment in the collection of placement and student 
assessment data (75% of those states responding). 

2. Planning and operational processes, employer 
feedback and special needs data appear to be 
collected and used by both SEA's and LEA's. 

3. There appears to be a lower i nvol veipent of 
SEA's in the collection of student assessment 
and pi acement. data when compared wi th otheY 

: evaluative data and with 'LEA involvement. 

4. The SEA's tend to be the heavier users of 
evaluative data, .except for student assess- 
ment data and for employer feedback data, 
which are used' equally by SEA's and LEA's. 



2. In your state, who collects, receives anjHises 
evaluation data required by federal legislation? 

err — - — • 

COt^ECTS RECEIVES USES 
SEA. LEA 



• Placement Data 

• Planning and 
Operational 
Processes Data 

• Employer Feedback 
Data 

• Special Needs 
Services Data 

• Student Assessment 
Data 



15 37 



27 
21 

21 
12 



26 
29 

32 
36 



SEA LEA 
41 16 



39 
40 

38 
24 



21 
18 

19 
26 



STTTLTA 

43 .37 

44 33 
39 i38 

38 32 
23 '35 



Key .Question 3 . What is the perceived utility of 
current evaluation practices in 
SEA's? 



conclusions. 



evidence 



.1. In general , -there is an indication that 
state directors are satisfied with their 
state's evaluation system*. 



2. Maying program improvements, preparing 
accountability reports, and aiding de- 
cision making are uses of evaluation 
results with the highest perceived 
success. 



3. Least success was reported as occurring, 
for informing the public and discontinu- 
ing programs. 



4. Supporting legislative request received 
the, most varied rating of all those in- 
cluded. 
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1. How satisfied are you with your state's system for 
evaluating- the quality and effectiveness of voca- 
tional education programs? (mean =.4.0, Sd = .94*). 

Not Satisfied " Very Satisfied 

1 . 2 3 4 • 5 , ■ 

— * 

5. How useful would you say are your state's evaluation 
activities? (mean = 4.1, Sd = .68) 

Not Useful Very Useful 
.. I - 2 3 4 5__ 

6. What success has been achieved in using evaluation 
results? 

Low 1 2 3 -o 4 5 High 



Make Improvements 
in programs 


mean 




411 


Sd- 


s 


.82 


Inform the Public 


mean 




2.8 


Sd 




1.02 


Aid Decision Making 


mean 




3.9 


Sd 




.64 


Discontinue Programs 


mean 




2.8 


Sd 




l!06 


Support Legislative 
Requests ' 


mean 




3.3 


.Sd 




1.20 


Prepare Accountability 
Reports 


mean 




4.1 


Sd 




.82 


Inform Legislators 


mean 




3.0 


Sd 




.92 


Add New Programs 


mean 




3.1 


Sd 




.94 



*Sd is the notation for standard deviation, a statistical 
measure of disperion or variance. 



• : : \ . . ' 

.Key Question 4. What "factors facilitate and deter 
the initiation and conduct of .. 
state evaluation practices? 




conclusions 



evidence 



1. Improvement of programs, meeting federal 
* mandates, and -improving planning were - 
% the highest ranked /reasons for^current 
state evaluation practices. • 

1 ' T* 



2. The' lowest ranked were fiscal pressures, 
and State Advisory Council for Vocational 
Education (SACVE) requests. 



3. The major factor ijn the success of state 
evaluation practice is perceived to be 
the SEA staff (commitment, attitude and 
expertise). LEA staff attitudes toward 
evaluation are also considered a contri- 
buting factor to success. Technical 
■ assistance provided by the federal 
government is perceived to be insignifi- 
cant. ► 
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7. 



8. 



What reasons led your state to adopt its current 
evaluation practices? Please rank in order of de- 
scending importance. Most important = (1), Least 
Important = (8). 

(Mean Rank) 

Meet Federal Mandate - 



.2.3 

4.4 , 

,5.2 

1,8 

3.0 

6.1 

4.7 

7.7 
(Frequency) 
36 

13 

2 

I 17 
34 

. 44 ■ 

15/ 

14 ' 

15 ' ' 
40 ' 



• / 



Meet State Legislative Mandate 
Advisory- Council Request 
Interest in Program improvement 
Need to Improve' Planning 
Need to Make Budget Cuts 
Need to Justify Activity / 
Other I 

Existing Evaluation Expertise of Staff 

Federal Guidelines 

Federal Technical Assistance 

Existing Documents (Books, Guides) 

Po'oftive Attitudes toward Evaluation by LEA 1 s 

Positive Attitudes toward Evaluation by 
SEA Staff 

Available Resources 
Success of Previous Evaluation 
Available Consultant Assistance 
Strong Commitment to Evaluation 



ERIC 
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• Key Question 4. (continued) 




conclusions 



evidence 



4 4. Inadequate resources appears to be 
the major obstacle to the develop- 
ment of evaluation activities. 



* 



9. What factors hampered the development of evaluation 
activities in your state? 



(Frequency) 
15 

12 

• 20 

13 - 

10 



26 
5 
6 
8 



Lack -of Evaluation Expertise 

Inadequate Federal' Guidelines » 

Lack of Federal Technical Assistance 

Lack of Guides .and Books 

Negative Attitudes toward Evaluation 
by LEA's 

Negative Attitudes toward Evaluation 
by SEA staff 

Inadequate Resources 

Failure of Previous Evaluation 

lack of Consultant Assistance 

Lack of Commitment to Evaluation 
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-Key Question 5 



Should the existing criteria, fdr 
evaluation be retained? 




conclusions 



evidence 



JL Employer reaction and student performance 
were the least desired evaluation components 
if federal ! requirements were eliminated; 
however, aimajority of states (66-87%) . 
would continue to use all five evaluation 
* components! 



i - 
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10. If federal requirements for evaluation were elimina- 
ted, which aspects of your state evaluation prdcess 
might be dropped? 

(Frequency) ' 

7 Placement ^ 

12 Employer Reaction 

8 Planning and Operational' Processes v , 

9 Services to Special Populations 
11 * Student Performance 

16 Hone 
1 Teachers & School Directors Performances 

11. If federal requirements for evaluation were elimin- 
ated, what would you choose to evaluate? 



V 



43 


PI acement 


35 


Employer Reaction ^ 


38 


planning and Operational Processes 


36 


Services to Special Populations 


34 


Student Performance 


1 


Total Program 


- 1 


None 



17 



erjc — 



> 



0 

♦ Key Question 6 . What is" the level of imple- 
mentation" of current evaluati 
practices? * 



conclusions 



.1. State evaluation activities were^careied 
out primarily by SEA staffs with cooperff= 
tive approaches used by 17%. 



2. Placement evaluation and employer assess- 
ment appear.to be the. most extensively 
" implemented evaluation activities with 
special needs services and student assess- 
ment existing at the lowest level of im- 
plementation. ..,-»• 




18 




evidence 



\ ■ m 

3\ Are your state evaluation act-tyi ties conducted: 
. (Frequency) * ' \ • - 

^^i-37 By your SEA staffs x v 
* l^-UJnder contract with an outside organi- 
zation 

7 Both ^^-^^ 
|1 Neither , ^-^^^ 

4. Please check the following evaluatio^etivUie*?- as 
to their level of implementation in your stateU^^ 

FULLY IN NOT 

IMPLEMENTED PROCESS STARTED 

• Placement Level of 

V Gjrads " 35= 15- 0 

• Employer Assessment 

ofGrads 27 20 0. 

• Assessment of Planning 
and Operational Pro- 



cedures ? . 


21 


24 


0 


* Assessment of Special 




> 




. Needs Services 


18 


-29 


0 


t Assessment of Student 








Performance 


14 


. 30 


0 








19 



* . / Key-Question 7, What is the*desired unit of 

sampling for evaluation? _ 

-* 

- • • . ' .ft 

conclusions v 



"The Wedominantly desired unit of sampling 
for evaluation appears to be individual pro 
grams A Although, for program improvement, 
attention would be giyen to both programs 
and institutions. 



•< . 20 



evidence • [ 

w • 

12. In your state, >f the goal was .improvement of 
vocational offerings, would the SEA direct its 
efforts toward evaluating: 

(Frequency) . . 

30 Individual, Programs 

10 Institutions 
9- Both 

1 Neither 

13. In your state, if the goal was to improve pro- 
grams, where would SEA attention.be directed? 

(Frequency) * % 

11 « Programs 

1 Institutions 

37 4 Both 

* 



21 



Kay Question 8. What suggestions do state di rectors' have- for • 

* writing new legislation with respect to evaluation? 

<\ * 

The following responses were made by respondents . - 

^l^X Provide more latitude for states to develop their own evaluation around 
.their needs (Frequency = 9). . , 

Provide funds for evaluation (Frequency = 7). 

3. Present provisions are acceptable (Frequency = 6). 

4. Eliminate student performance (Frequency. = 3). 

5. Eliminate employer satisfaction as a state requirement (Frequency 2). 

6. Lessen prescriptiveness of- rules and regulations (Frequency = 2). 

7. Base" evaluation on program results: . , * 

Job satisfaction 

employer ratings . ♦ 
. — . job performapce „ 

' 8. Eliminate 1 year follow-up and replace with 5 and 10 year follow-up. 

\ 

9. Delete planning and operational processes. 

10. Maximize sampling in employer-survey. 

11. Expand criteria beyond placement, and -employer satisfaction. 

12. Provide -time for developing student competency data. 

13. Relegate employer feedback jto national research. 

14. Develop student tests" nationally. 

15. Require evaluation techniques*for all parts of law. 

16. Reduce data burden. 

- \ 

17. Develop guidelines 9 months prior to reporting. \ 



18. Report data for 5 years. 

19. -Incorporate VEDS and evaluation ■ 

20. - Make evaluation program improvement oriented. 
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Key Question 8, (Continued) 

21« Place emphasis on the us£of data. 

22. Change special population evaluation f*om program to school. 

23. Expanf'OE codes to include, broader occupations. 

24. Rewrite planning and reporting* requirements. 
/25. Require public participation. , 

26., Urge coordination with regional accrediting associations. 

27. * ■ Encourage a continuous evaluation by LEA's. 

28. Fund VEDS. 

29". Require evaluation for all Voc. Ed. programs regardless of funding. 

30. Provide technical- assistance on evaluation. 

31. Evaluation only federally funded activities. 

32. Make, accountability more product -oriented. 

33. Promote qualitative measurement of programs. , 

9 

* 34. Describe the technical assistance role to be carried out for program 

* improvement. 

35. Fire developers of guidelines. 

36. Provide Targe set asides for territories. 

37. Gain input from practitioners. 
38. . Provide block grants. • 

39. Continue state plan process. 

40. Eliminate NOICC/SOICC. 

41. Mandate fewer "must haves" on SACVE. " 

42. . Combine sex equity with guidance section with funding. 

43. Reestablish 'regional offices. 

44. Add specificity to legislative purpose programs and special populations 
services. 

45. Eliminate 104.402 (d) (Frequency. = 6). 

* 

o • 46. Provide for;, uniform procedures for conducting evaluation of programs, 
ERXC institutions, student performance and others. 



discussion 

0- 

m » 

. This report provides an added dimension to the two* previous observa- 

* * 

tions of evaluation in vocational education made by Contract Research . 
Corporation in, 1978 and by Abt Associates in^the 1979-80 .school year. Also, 
this report adds to the evaluation section of the interim report of the 
vocational education study (NIE, 1980) and- an-excel lent paper entitled, ~ 
"Evaluating Vocational Education: The Federal Stimulus" (Hendrickson, 
1981). * ... 

This study indicates .that a high level of evaluation activity exists 
among the states. All states (of those responding) indicate that they have 
either fully implemented or ,are in the process of implementing the mandated 
evaluation activities. * 

The following paragraphs reflect the /researcher's perception of 
several factors associated with vocational education evaluation in the states. 
These factors include:, purpose of evaluation, focus of evaluation, imple- . % 
mentation of evaluation and use of evaluation results. \' 
Purpose of Evaluation 

States indicate that program improvement is the primary reason for 
adopting their current evaluation practices. This finding supports the 
finding of the two previous observations by CRC and Abt. Meeting the 
federal mandate and'desiring to improve planning were rated second and 
third, respectively. This finding appears to imply that the. federal intent 
of improved planning and program improvement has been transmitted, to and 
internalized by state education agencies. The researcher supports the 
notion that program improvement should be the primary purpose of program 
evaluation. - ' "\. 

: ' ' . - 14 
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Focus of Evaluation » , ' 

Employed feedback and placement of former students are identified as 
the most often identified indicators of program quality (47 and 44 respectively 
from 50 respondents)'. However, when asked about changes in evaluation 
practices if federal evaluation requirements were abolished, 12 respondents - - 
indicated that they would eliminate employer reaction and 7 would eliminate 
placement .assessment. It- could be that respondents may believe in &e 
utility and validity of employer feedback,, but some. .may not have confidence 
.in the process that is currently used. .That is., many believe that employers 
provide valuable input but,surveys with minimally responding groups may be in- 
significant. This finding may speak to policy makers regarding 'jthe mappropn- 
ateness of specifying proced'ires without tryout and utility testing. 

It is also interesting to note that several criteria, other then those 
mandated by legislation and regulations, are viewed as important by, SEA; s*. 
For example, job satisfaction (frequency = 37), were cited as indicator of 
program quality. Perhaps evaluation should not be limited to the traditional 
(legislated) criteria. It may be beneficial to identify the intercorrelation 
of various process and proauct measures (since most states do process tvaluflgK. 
tions fairly well, and they relate to their technical assistance role) of 
placement, employer reaction, and job satisfaction. The question is, "Why 
collect all this information when a portion of it may tell you the same 
thing?" For example, student evaluation of instruction results (end of ( 
* course, feedback) may be highly correlated with student follow-up results. 
Implementation of Evaluation ' - • • 

As previously mentioned, an extensive amount of activity has occurred 
within the states to implement required evaluation activities. It can also 



by s«fn, by reviewing the earlier observational reports (CRC and Abt) that 

much of this activity has resulted since the passage of the Education 

"Amendments of 1976 (PL 94-482), This fact speaks favorably, for federal 

* . t - • • 

•action having impact on states and for states being responsive. 

Slightly different f ronr the Abt findings in 15 states, student follow- 

up and employer assessment of former students are the most fully implemented 

- i " 

evaluation activities (35 states for placement and 27 states for employer 

■ / ' - • ■ - * 

assessment compared to 21 states fbr planning and operational processes). 

Assessment of student 'performance and assessment of special needs services 

remain^ the least implemented activities. 

• The findings of. this survey .portray a cooperative effort in the collection 

and use of evaluative information^ The collection pf information, appears 

* * * * ' ■ 

to be a local responsibility in- about half the states with .the exception of 
placement data which is collected by LEA's in more than half the states. 
Use of results appear to be an activity in which both SEA'S and LEA's are 
heavily involved. 

. It is interesting to note that respondents were generally satisfied 
with their state's system for evaluating vocational programs. This may 
provide- an indication that the, evaluation activities have been internalized # 
and are not being viewed solely as a reaction to. a federal mandate. It may 
also mean that the states are patting themselves on the back. 
Use of Results 

\ ' The-use of ev&luation^results is a necessary step in achieving the pur- 
pose^) of evaluation. Respondents indicated a high level of usefulness of 
their evaluation activities (X"« 4.1 on a scale of 1-5). When asked to rate 

the success .of using evaluation results for specific purposes, making improve : 

— * 

■ents-'OT* 4.1), sparing accountability reports (X = 4.1), and aiding 



decisionmaking 0T = 3,.9) were the highest. The use of results in "supporting 
legislative request" has the greatest range of success. That is;" some had 
high success and 'some had very little success. It can be speculated that 
various ty^es of results, e.g., student follow-up, planning and operational 
processes have different uses. Also, it is probable that some SEA persons 
have more experience than others in the.use of results. This step in the 
evaluation process, in the researcher's' mind, needs greater attention — 
procedure development, staff development and technical assistance. . 

general recommendation^ ? 

Thr foil owing statements are broad recommendations for the further de- 

Velopment of .federal policy related to the evaluation of, vocational education. 

" ? . * 

These, it js. thought*, should be the framework for any specific legislative re- 

. , • i* 

quirements. * 

1. -Continue to emphasize the improvement function of 

evaluation and further develop ways of facilitating 
the use of results for improvement. 

2. Analyze extant research findings and conduct new 
- research' on the validity, of process measures for 

predicting products or outcomes of 'vocational ed- 
ucation, Thdse findings should provide the base 
for determining evaluative criteria. 

\ 3.* The requirements for evaluation procedures should 
, remain somewhat consistent to minimize state and 
-'local burden-and to provide for the enhancement of 
currently used procedures. 

• 

4. Any new approaches should be tried out prior to the 
preparation and issuance of rulesTand regulations. . 

5; The, use of evaluation results should receive added 

emphasis in - terms of new procedures, staff development, 
-** ■ and technical assistance. , - x .. 



a suggestion 

The evaluation of vocational education program! is a worthwhile in- 
vestment of tax generated funds* If this statement is accepted, then policy 
makers must^trive to maximize the return on dollars invested through in- 
formed decisions and established practice. Also, it is important to strive 
for a certain amount of stability in policy. That is, it is? obvious that 
many PL 94-482 requirements are only recently b.eing fully implemented. To 
change drastically at this time may frustrate and deter rather than build 
and enhance. The suggestion being posed herein is focused, on maintaining 
consistency in evaluation intent and polishing and honing existing evalu- 
ation procedures and steps for utilizing results. . % 
Rationale ' 

— — — — — — — * b 

t 

Program improvement has been a major intent of federal legislation 
in vocational education. The use of information, (e.g., labor force data, 
evaluation data), was identified as one means of facilitating program im- 
provement. Though a worthy expectation, the practice of evaluation and the 
use of its results have . fall en far short of the expectation. Many evalua- 
tive activities such as employer surveys and student assessment have been 
viewed more as compliance oriented than improvement driented. Consequently, 
the resuTts therefrom have not been maximally used for improvement. This 
is a case of appropriate intent, possibly appropriate procedures, but in- 
appropriate use. . o 

.Furthermore, many of the required or federally chosen criteria for 
evaluating vocational programs have been deemed inappropriate. For example, 
many have argued that employment is outside the realm of control of vocational 
education, since the economy and other forces are involved. Also, inputs to 
programs— students— may vary considerably and may effect outcomes significantly 

28 
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' Procedurally, evaluation of vocational programs suffer from 'technical 
and.staff implementation problems. Response rates' to employer surveys and 
I technology deficiencies are butjaro basic methodological problems. 
In additiion/th>4iiconsistencies/ that exist within and among states in the 
collection of evaluative data-pose^ severe problems in aggregating informa- 
tion. Al\l this leads to a 'need for impfovemejrt. ■ 
The Suggested Approach , . . 

The ^improvement function of evaluation should be held high and existing 
-eval uati on\ef forts should be enhanced. This can be accomplished^ by holding 
educational! agencies (local and state) accountable' for improvement,' That 
is, LEA's should be given the responsibility to document changes' that have 
occurred in Aheir vocational programs. These changes need not be limited 
to growth but\ should focus on qualitative changes as well. For example, 



0 



process. Or 
local use., 

. Reporti 
should be on 



the elimination of a program could, be a positive change for an institution. 

•Evaluation techniques should be used^wi thin this, thrust to aid-in 
making improvements and changes *as well as aiding in their documentation. 
LEA's could |hoole the appropriate evaluative techniques to use in the 
SEa\s could specify a variety of legitimate techniques for 

' ■ ' ■'• 

ng cC"Uld be done in some consistent format, but the emphasis 
using! results' to improve rather than on reporting to an ex- 
ternal party.1 This approach will encourage agencies, to actually initiate 
changes rathel than to simply report. Additionally, results could be 'used 
by agencies for other purposes as well. \ ' 

A Supplement , \ 

In addition to! this change/improvement evaluation approach,, it may be 
"useful to incorporatl periodic statewide and national surveys' to provide 



29 



19 



• - t needed gross information" for state and federal policy making 
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"A NEW WAY" / 
FOR EVALUATING VOCATIONAL /EDUCATION 



TITLE 
•PURPOSE 

SCOPE 



PROCEDURE* 



TIMEFRAME 



Evaluation For and Of/ Improvement . 

1 ' ' ■ 

To facilitate and document change/improvement 
in vocational programs. Al.so,. future develops 
ment of national policy could be enhanced. 

Local Programs and /institutions; state; and 
federal . 



EVALUATIVE QUESTIONS: . 1. 



What changes^ nave occurred in the delivery 
of vocational^ education? 

. What' stimuli j are responsible for program 
changes? . 



2. 



1. Each LEA will evaluated ts vocational 
programs using self -selected or SEA-, 
selected techniques. 

2i. Results will be reported to the SEA re- 
garding the specifics of change and the 
facto rs. contributing to the change. 

3. SEA's will summarize the type and extent 
of local change and. report it to the -' : 
Department of Education. 

4. A series of periodic national surveys and/or 
case studies will be sponsored to proyide 

a consistent data base for necessary federal 
policy development. s ^ 

Could vary. Might require continuous LEA evalua- 
tion with third year reporting of change. 
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